Distributed support vector machine privacy-preserving method, system, storage medium and application

ABSTRACT

A distributed support vector machine privacy-preserving method includes: dividing a secret through secret sharing among all participating entities, iteratively exchanging a part of the information divided by the participating entities, and solving sub-problems locally; performing an iteration until a convergence is reached to find a global optimal solution; and in consideration of the generality of the privacy-preserving method, adopting a privacy-preserving method based on a vertical data distribution and a privacy-preserving method based on a horizontal data distribution, respectively; wherein the participating entities do not trust each other, and interact through a multi-party computation for local training. The method is applied to an honest-but-curious scenario, and uses the idea of data division to perform local computation through the interaction of part of the data among users to finally reconstruct the secret to preserve data privacy.

CROSS REFERENCE TO THE RELATED APPLICATIONS

This application is based upon and claims priority to Chinese PatentApplication No. 202110054339.X, filed on Jan. 15, 2021, the entirecontents of which are incorporated herein by reference.

TECHNICAL FIELD

The present invention pertains to the technical field of dataprivacy-preserving, and more particularly, to a distributed supportvector machine privacy-preserving method, system, storage medium andapplication.

BACKGROUND

Today's information age is now witnessing the explosive growth of data.As the scale of computer systems becomes larger and larger, distributedprocessing methods are increasingly accepted by the related industry.Moreover, machine learning algorithms have been applied to variousfields. In this case, since the distributed processing method can handlea larger amount of samples, it can better exploit the advantages ofmachine learning algorithms so that the algorithms can be applied on alarge scale. The support vector machine is one of the most widely usedmachine learning algorithms. Prior studies generally use alternatingdirection method of multipliers (ADMM) algorithms to solve machinelearning optimization problems such as optimization problems of supportvector machines. Meanwhile, data used for training are owned by multipleentities, but the sharing and training of the data are hindered due tothe sensitivity of the data. Most distributed algorithms require eachnode to explicitly exchange and disclose state information to itsneighboring node in each iteration. This means that many practicaldistributed applications face serious privacy issues. In this regard, itis unacceptable to merely save the original data locally forprivacy-preserving, and it is also necessary to preserve the privacy ofthe interactive parameters in the process of implementing thedistributed ADMM algorithm. Herein, an ADMM algorithm-basedprivacy-preserving technique will be presented based on a support vectormachine scenario.

However, existing research on privacy-preserving of support vectormachine scenarios still faces challenges to be urgently solved in termsof privacy and accuracy. Two methods are commonly used to realizeprivacy-preserving in distributed optimization algorithms. The firstmethod is a perturbation method, which mainly uses the technique ofdifferential privacy. This method is highly efficient, but introducesnoise and thus will cause the loss of data availability and impair theaccuracy of the optimization results. Although the related studies havemade a balance between privacy and accuracy, the speed of convergence tothe optimal classifier will always slow down. The second method is acryptographic method, including secure multi-party technology andhomomorphic encryption. The homomorphic encryption method hasexcessively high computational overhead and is thus difficult to applyto practical applications. Additionally, in the current research, mostof the support vector machine privacy-preserving scenarios involve onlydistributed deployment of data and single-machine processing withoutconsidering the privacy leakage problem of information interactionduring the collaborative training of the fully distributed supportvector machine algorithm with multiple machines and multiple datasources. Some research work has focused on the privacy leakage problem,but has not fully resolved the horizontal and vertical distribution ofdata.

As analyzed above, the prior art has the following problems andshortcomings. The existing distributed support vector machines havemutually reciprocal shortcomings between computational overhead andsecurity. That is, a high-security method has the problem of highcomputational overhead, whereas a high-efficiency method has securityissues. In addition, such methods must give consideration to both themachine learning scenario and the accuracy of the training results.

The difficulty of solving the above-mentioned problems and shortcomingsis: to solve the privacy problem of the interactive computation of theintermediate states in the machine learning training process. Althoughhomomorphic encryption can perform multi-party secure computation, itincurs high computational complexity.

To solve the above-mentioned problems and shortcomings, it is highlydesirable to provide a high-efficiency method capable of simultaneouslyensuring the security of multi-party computation when processing datafor machine learning training to achieve the same effectiveness ashomomorphic encryption without substantial overhead, thereby preservingdata privacy based on the premise of an ensured accuracy of the trainingresults.

SUMMARY

In view of the problems in the prior art, the present invention providesa distributed support vector machine privacy-preserving method, system,storage medium and application.

The present invention is achieved by adopting the following technicalsolutions. A distributed support vector machine privacy-preservingmethod includes: dividing a secret through a secret sharing among allparticipating entities, iteratively exchanging a part of informationdivided by the participating entities, and solving sub-problems locally;performing an iteration until a convergence is reached to find a globaloptimal solution; and in consideration of the generality of theprivacy-preserving method, adopting a privacy-preserving method based ona vertical data distribution and a privacy-preserving method based on ahorizontal data distribution, respectively; wherein the participatingentities do not trust each other, and interact through a multi-partycomputation for local training.

Further, the distributed support vector machine privacy-preservingmethod specifically includes:

step 1: establishing a network communication environment with aplurality of data sources;

step 2: choosing a support vector machine scenario with a verticaldistribution or a horizontal distribution according to a datadistribution of the data sources;

step 3: allowing all participating entities to solve the sub-problemslocally;

step 4: allowing all participating entities to use a Boolean sharing tosplit a penalty parameter and exchange a part of the penalty parameterwith a neighboring node to update the parameter;

step 5: allowing all participating entities to use an arithmetic sharingto split an updated iterative variable and exchange a part of theupdated iterative variable with the neighboring node to compute aLagrange parameter in a shared form;

step 6, allowing all participating entities to reconstruct the secret;

step 7, returning to step 3 if the iteration does not reach theconvergence; and

step 8, outputting a training result.

Further, an objective function of the horizontal distribution and anobjective function of the vertical distribution in step 2 arerespectively:

${\min_{v_{i},}{\xi_{i}\frac{1}{2}{\sum\limits_{i = 1}^{N}{{v_{i}^{T}( {I_{D + 1} - \Pi_{D + 1}} )}v_{i}}}}} + {NC{\sum\limits_{i = 1}^{N}{1_{i}^{T}\xi_{i}}}}$$s.t.\mspace{14mu}\{ \begin{matrix}{{{Y_{i}B_{i}V_{i}} \succcurlyeq {1_{i} - \xi_{i}}},{{\xi_{i} \succcurlyeq {0_{i}( {{i = 1},\ldots\mspace{14mu},N} )}};}} \\{{AV} = 0}\end{matrix} $

and

wherein v_(i)=[ω_(i) ^(T),b_(i)]^(T), V=[v₁ ^(T), . . . v_(N) ^(T)]^(T),B_(i)=[X_(i),1_(i) ^(T)], 1_(i)∈R^(1*M), X_(i) is an i^(th) participantand a j^(th) participant, X_(i)=[[x_(i1) ^(T), . . . x_(iM) ^(T)]^(T)],Y_(i)=diag(y_(i1), . . . , y_(iM)), y_(ij) is a j^(th) data entry and acorresponding label, ξ_(i)=[ξ_(i1), . . . , ξ_(iM)], N is a number ofparticipants, and M is a number of training set samples for eachparticipant of the participants; and

${\min{\sum\limits_{i = 1}^{N}{\frac{1}{2}{v_{i}^{T}( {I_{D_{i} + 1} - \Pi_{D_{i} + 1}} )}V_{i}}}} + {1_{M}^{T}( {1_{M} - {Y{\sum\limits_{i = 1}^{N}{B_{i}v_{i}}}}} )_{+}}$s.t.  z_(i) = B_(i)v_(i), i = 1, …  ,;

wherein v_(i)=[ω_(i) ^(T),b_(i)]^(T), B_(i)=[X_(i),1_(M)],1_(M)∈R^(M*1), Y_(i)=diag(y₁, . . . ,Y_(M)), y_(j) is the j^(th) dataentry and the corresponding label.

Further, iterative processes of solving the sub-problems locally in step3 are respectively:

horizontal data distribution:

${\{ {v_{i}^{k + 1},\xi_{i}^{k + 1}} \} = {{\arg\;{\min_{v_{i},\xi_{i}}{L( {v_{i}\xi_{i}\rho^{k}\lambda^{k}} )}}} + {\frac{r_{i}}{2}{{v_{i} - v_{i}^{k}}}^{2}}}}, \rho^{k}arrow\rho^{k + 1} ,{{\lambda_{i,{i + 1}}^{k + 1} = {\lambda_{i,{i + 1}}^{k} + {\rho_{i,{i + 1}}^{k + 1}( {v_{i}^{k + 1} - v_{i + 1}^{k + 1}} )}}};}$

and

vertical data distribution

${v_{i}^{k + 1} = {{\arg\;{\min_{v_{i}}{f_{i}( {vi} )}}} + {\frac{\rho}{2}{{{B_{i}v_{i}} - {B_{i}v_{i}^{k}} - {\overset{¯}{z}}^{k} + {\overset{\_}{Bv}}^{k} + u^{k}}}^{2}}}},{{\overset{¯}{z}}^{k + 1} = {{\arg\;{\min_{\overset{¯}{Z}}{g( {N\overset{¯}{z}} )}}} + {\frac{N\;\rho}{2}{{\overset{¯}{z} - {\overset{\_}{Bv}}^{k + 1} - u^{k}}}^{2}}}},{u^{k + 1} = {u^{k} + {\overset{\_}{Bv}}^{k + 1} - {{\overset{¯}{z}}^{k + 1}.}}}$

Further, a method of using the Boolean sharing to split the penaltyparameter in step 4 specifically includes: considering ρ^(k)→ρ^(k+1) isa progressive increase and an upper bound is r_(i), obtaining anappropriate value through a comparison to update ρ, and dividingρ_(i, i+1) ^(k) into ρ_(i, i+1) ^(k)=q_(i, i+1) ^(k)+q_(i+1, i) ^(k) tosecurely compute ρ_(i, i+1) ^(k); wherein an i^(th) participant providesq_(i, i+1) ^(k) and q_(i, i+1) ^(k+1), and an (i+1)^(th) participantprovides q_(i+1, i) ^(k) and q_(i+1, i) ^(k+1); comparing q_(i, i+1)^(k)+q_(i+1, i) ^(k) with q_(i, i+1) ^(k+1)+q_(i+1, i) ^(k+1) withoutexposing q_(i, i+1) ^(k), q_(i+1, i) ^(k), and q_(i+1, i) ^(k+1),converting each term into a Boolean type, and performing a secureaddition and comparison by using a Yao's garbled circuit.

Further, a method of using the arithmetic sharing to split the penaltyparameter in step 5 specifically includes: in a (k+1)^(th) iteration,securely computing

ρ_(i, i + 1)^(k + 1)(v_(i + 1)^(k + 1) − v_(i)^(k + 1))  as  (q_(i, i + 1)^(k + 1) + q_(i + 1, i)^(k + 1))(v_(i + 1)^(k + 1) − v_(i)^(k + 1))

by using Shamir's secret sharing, and arithmetically dividing each term,wherein an i^(th) participant provides

q_(i, i+1) ^(k+1)

₁ ^(A),

q_(i, i+1) ^(k+1)

₂ ^(A),

−v_(i) ^(k+1)

₁ ^(A),

−v_(i) ^(k+1)

₂ ^(A),

−q_(i, i+1) ^(k+1)v_(i) ^(k+1)

₁ ^(A), and

−q_(i, i+1) ^(k+1)v_(i) ^(k+1)

₂ ^(A), an (i+1)^(th) participant provides

q_(i+1, i) ^(k+1)

₁ ^(A),

q_(i+1) ^(k+1)

₂ ^(A),

v_(i+1) ^(k+1)

₁ ^(A),

v_(i+1) ^(k+1)

₂ ^(A),

q_(i+1, i) ^(k+1)v_(i+1) ^(k+1)

₁ ^(A), and

q_(i+1, i) ^(k+1)v_(i) ^(k+1)

₂ ^(A), the i^(th) participant sends

q_(i, i+1) ^(k+1)

₂ ^(A),

−v_(i) ^(k+1)

₂ ^(A) and

−q_(i, i+1) ^(k+1)v_(i) ^(k+1)

₂ ^(A) to the (i+1)^(th) participant, the (i+1)^(th) participant sends

q_(i+1, i) ^(k+1)

₁ ^(A),

v_(i+1) ^(k+1)

₁ ^(A) and

q_(i+1, i) ^(k+1)v_(i+1) ^(k+1)

₁ ^(A) to the i^(th) participant, and the i^(th) participant locallycomputes

−q_(i+1, i) ^(k+1)v_(i) ^(k+1)

₁ ^(A) and

q_(i, i+1) ^(k+1)v_(i+1) ^(k+1)

₁ ^(A) to finally determine the value of (q_(i, i+1) ^(k+1)+q_(i+1, i)^(k+1))(v_(i+1) ^(k+1)+v_(i) ^(k+1)) in the shared form as

(q_(i, i+1) ^(k+1)+q_(i+1, i) ^(k+1))(v_(i+1) ^(k+1)−v_(i) ^(k+1))

₁ ^(A)=

q_(i, i+1) ^(k+1)v_(i+1) ^(k+1)

₁ ^(A)+

q_(i+1, i) ^(k+1)v_(i+1) ^(k+1)

₁ ^(A)+

−q_(i+1, i) ^(k+1)v_(i) ^(k+1)

₁ ^(A)+

−q_(i, i+1) ^(k+1)v_(i) ^(k+1)

₁ ^(A); and similarly, the (i+1)^(th) participant computes

(q_(i, i+1) ^(k+1)+q_(i+1, i) ^(k+1))(v_(i+1) ^(k+1)−v_(i) ^(k+1))

₂ ^(A)=

q_(i, i+1) ^(k+1)v_(i+1) ^(k+1)

₂ ^(A)+

q_(i+1) ^(k+1)v_(i+1) ^(k+1)

₂ ^(A)+

−q_(i+1, i) ^(k+1)v_(i+1) ^(k+1)

₂ ^(A)+

−q_(i+1) ^(k+1)v_(i) ^(k+1)

₂ ^(A)+

−q_(i, i+1) ^(k+1)v_(i) ^(k+1)

₂ ^(A).

Further, a method of reconstructing the secret in step 6 specificallyincludes: allowing both parties to reconstruct the secret (q_(i, i+1)^(k)+q_(i+1, i) ^(k))(v_(i+1) ^(k)−v_(i) ^(k)) as (q_(i, i+1)^(k)+q_(i+1, i) ^(k))(v_(i+1) ^(k)−v_(i) ^(k))=

((q_(i, i+1) ^(k)+q_(i+1, i) ^(k))(v_(i+1) ^(k)−v_(i) ^(k))

₁ ^(A)+

(q_(i, i+1) ^(k)+q_(i+1, i) ^(k))(v_(i+1) ^(k)−v_(i) ^(k))

₂ ^(A), and compute λ_(i, i+1) ^(k+1)=λ_(i, i+1) ^(k)+ρ_(i, i+1)^(k+2)(v_(i) ^(k+1)−v_(i+1) ^(k+1)) to update λ.

Another objective of the present invention is to provide acomputer-readable storage medium, wherein a computer program is storedin the computer-readable storage medium. When the computer program isexecuted by a processor, the processor executes the following steps:dividing a secret through a secret sharing among all participatingentities, iteratively exchanging a part of information divided by theparticipating entities, and solving sub-problems locally; performing aniteration until a convergence is reached to find a global optimalsolution; in consideration of the generality of a privacy-preservingmethod, adopting a privacy-preserving method based on a vertical datadistribution and a privacy-preserving method based on a horizontal datadistribution, respectively. The participating entities do not trust eachother, and interact through a multi-party computation for localtraining.

Another objective of the present invention is to provide a distributedsupport vector machine privacy-preserving system for implementing thedistributed support vector machine privacy-preserving method. Thedistributed support vector machine privacy-preserving system includes:

an information preprocessing module, configured for dividing a secretthrough secret sharing among all participating entities, iterativelyexchanging a part of information divided by the participating entities,and solving sub-problems locally;

an information iterative processing module, configured for performing aniteration until a convergence is reached to find a global optimalsolution; and

a privacy-preserving module, configured for adopting aprivacy-preserving method based on a vertical data distribution and aprivacy-preserving method based on a horizontal data distribution,respectively. The participating entities do not trust each other, andinteract through a multi-party computation for local training.

Another objective of the present invention is to provide a distributedsupport vector machine for implementing the distributed support vectormachine privacy-preserving method.

By means of the above technical solutions, the present invention has thefollowing advantages. According to the present invention, the supportvector machine for privacy-preserving is trained by combining an ADMMalgorithm and the secret sharing. During a training process among theentities, the entities exchange part of the information divided bythemselves for collaborative training. The present invention is based onan honest-but-curious model, in which all participating entities do nottrust each other, and complete the training under the premise thatindividual information will not be leaked. Compared with the dataprocessing method based on homomorphic encryption, the present inventionhas the features of simple computation and low computational overhead.Compared with the differential privacy method, the present inventionprovides cryptographically strong and secure privacy-preserving withoutaffecting the accuracy of the training result.

TABLE 1 Comparison between efficiencies of a multi-party securecomputation scheme and a homomorphic encryption scheme FSCM-C D QOffline Online ADMIVI-Paillier FSVM-S 100 100 0.214 2.76E−4 0.06982.19E−4 200 200 0.810 3.10E−4 0.155 2.75E−4 300 300 1.83 3.33E−4 0.2172.92E−4 400 400 3.24 3.70E−4 0.297 3.07E−4 500 500 5.13 4.01E−4 0.3623.20E−4 600 600 7.46 4.16E−4 0.432 3.32E−4 700 700 10.02 4.53E−4 0.5093.44E−4 800 800 13.21 4.82E−4 0.615 3.67E−4 900 900 16.60 5.08E−4 0.6473.79E−4 1000 1000 20.56 5.35E−4 0.722 3.98E−4

In the present invention, the distributed support vector machineprivacy-preserving method is based on a secure multi-party computationand an ADMM algorithm and, in an honest-but-curious scenario, uses theidea of data division to perform local computation through theinteraction of part of the data among users, thereby finallyreconstructing the secret to preserve data privacy. Since the whole datavalue of a single user is related to privacy information, after the datais divided, each collaborative user is allocated a part of the data forlocal computation. In this way, the partners entirely cannot get therelevant privacy information of other users, and all information withexplicit semantics which the partner may obtain is only its own valueand the final computed result.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to explain the technical solutions of the embodiments of thepresent invention more clearly, the drawings used in the embodiments ofthe present invention will be briefly introduced below. Obviously, thedrawings described below are only some embodiments of the presentinvention. Those of ordinary skill in the art can obtain other drawingsbased on these drawings without creative efforts.

FIG. 1 is a flow chart of a distributed support vector machineprivacy-preserving method according to an embodiment of the presentinvention.

FIG. 2 is a structural schematic diagram of a distributed support vectormachine privacy-preserving system according to an embodiment of thepresent invention, wherein 1 represents information preprocessingmodule; 2 represents information iterative processing module; and 3represents privacy-preserving module.

FIG. 3 is a schematic diagram of an application scenario according to anembodiment of the present invention.

FIG. 4 is a schematic diagram of the implementation principle of thedistributed support vector machine privacy-preserving method based onsecure multi-party computation according to an embodiment of the presentinvention.

FIG. 5 is a first schematic diagram of the collaborative training of twonodes of a breast cancer data set according to an embodiment of thepresent invention.

FIG. 6 is a second schematic diagram of the collaborative training ofthe two nodes of the breast cancer data set according to an embodimentof the present invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS

In order to make the objectives, technical solutions, and advantages ofthe present invention clearer, the present invention will be furtherdescribed in detail below with reference to the embodiments. It shouldbe understood that the specific embodiments described herein are onlyused to explain the present invention, rather than to limit the presentinvention.

In view of the problems in the prior art, the present invention providesa distributed support vector machine privacy-preserving method, system,storage medium and application. The present invention will be describedin detail below with reference to the drawings. Herein, local supportvector machine sub-problems are solved by using a gradient descentmethod. Since the gradient descent method has a slow convergence rateand may converge to a local optimal solution, it may be replaced withimproved methods such as damped Newton's method and variable metricmethod to solve the local sub-problems. In consideration of the realscenario, different entities can use different methods to solve thelocal sub-problems.

As shown in FIG. 1, according to the present invention, a distributedsupport vector machine privacy-preserving method includes the followingsteps:

S101: a network communication environment with a plurality of datasources is established;

S102: a support vector machine scenario with a vertical distribution ora horizontal distribution is chosen according to a data distribution ofthe data sources;

S103: all participating entities solve the sub-problems locally by usinga gradient descent method;

S104: all participating entities use Boolean sharing to split a penaltyparameter and exchange a part of the penalty parameter with aneighboring node to update the parameter;

S105: all participating entities use arithmetic sharing to split theupdated iterative variable and exchange a part of the updated iterativevariable with the neighboring node to compute a Lagrange parameter in ashared form;

S106, all participating entities reconstruct the secret;

S107, returning to S103 if the iteration does not converge; and

S108, a training result is output.

Those of ordinary skill in the art can also implement the distributedsupport vector machine privacy-preserving method by using other steps.FIG. 1 only illustrates a specific embodiment of the distributed supportvector machine privacy-preserving method of the present invention.

As shown in FIG. 2, according to the present invention, a distributedsupport vector machine privacy-preserving system includes:

the information preprocessing module 1, configured for dividing a secretthrough secret sharing among all participating entities, iterativelyexchanging a part of the information divided by the participatingentities, and solving sub-problems locally;

the information iterative processing module 2, configured for performingan iteration until a convergence is reached to find a global optimalsolution; and

the privacy-preserving module 3, configured for adopting aprivacy-preserving method based on a vertical data distribution and aprivacy-preserving method based on a horizontal data distribution,respectively; wherein the participating entities do not trust eachother, and interact through a multi-party computation for localtraining.

The technical solutions of the present invention will be furtherdescribed below with reference to the drawings.

According to the present invention, the distributed support vectormachine privacy-preserving method includes: dividing a secret throughsecret sharing among all participating entities, iteratively exchanginga part of the information divided by the participating entities, andsolving sub-problems locally; performing an iteration until aconvergence is reached to find a global optimal solution; and inconsideration of the generality of the privacy-preserving method,adopting a privacy-preserving method based on a vertical datadistribution and a privacy-preserving method based on a horizontal datadistribution, respectively; wherein the participating entities do nottrust each other, and interact through a multi-party computation forlocal training.

As shown in FIG. 3, the application scenario of the present invention isa training process of a fully distributed multi-data source supportvector machine, and the network includes users participating in thetraining. According to the network topology, the users who need toparticipate in the training set an initial value. In the iterativeprocess, the value that needs to be computed collaboratively is dividedthrough secret sharing and then exchanged for computation. Since thedivided data cannot get any intermediate-state privacy information, dataprivacy and security are preserved.

As shown in FIG. 4, according to an embodiment of the present invention,the distributed support vector machine privacy-preserving method basedon secure multi-party computation specifically includes the followingsteps:

Step 1: a network communication environment is established, and anetwork topology situation where multiple users are adjacent to eachother is considered when the number of different users is set.

Step 2: the iterative processes of solving the objective function fortraining the support vector machine are determined according to the datadistribution of the data sources.

Step 3: in the (k+1)^(th) iteration, the user first updates v_(i) ^(k+1)according to the penalty function ρ and the Lagrange coefficient λupdated in the k^(th) iteration.

Step 4: in the (k+1)^(th) iteration, the user updates the penaltycoefficient ρ by taking the progressive increase as a constraintcondition. ρ_(i, i+1) ^(k) is divided into ρ_(i, i+1) ^(k)=q_(i, i+1)^(k)+q_(i+1, i) ^(k) to securely compute ρ_(i, i+1) ^(k) The i^(th)participant provides q_(i, i+1) ^(k) and q_(i, i+1) ^(k+1), and the(i+1)^(th) participant provides q_(i+1, i) ^(k) and q_(i+1, i) ^(k+1)q_(i, i+1) ^(k)+q_(i+1, i) ^(k) is compared with q_(i, i+1)^(k+1)+q_(i+1, i) ^(k+1) without exposing q_(i, i+1) ^(k), q_(i+1, i)^(k), q_(i, i+1) ^(k+1), and q_(i+1, i) ^(k+1). Each term is convertedinto a Boolean type, and is securely added and compared by using a Yao'sgarbled circuit. One party encrypts the truth table, one party performscircuit computation, and finally the secret is reconstructed. In thisway, an appropriate penalty coefficient ρ is determined.

Step 5: in the (k+1)^(th) iteration, the user solves the Lagrangecoefficient λ_(i, i+1) ^(k+1) by using the updated v_(i) ^(k+1) andρ_(i, i+1) ^(k), and securely computes ρ_(i, i+1) ^(k+1) (v_(i+1)^(k+1)−v₁ ^(k+1)) as (q_(i, i+1) ^(k+1)+q_(i+1, i) ^(k+1))(v_(i+1)^(k+1)−v_(i) ^(k+1)) by using Shamir's secret sharing. Each term isarithmetically divided. The i^(th) participant provides

q_(i, i+1) ^(k+1)

₁ ^(A),

_(i, i+1) ^(k+1)

₂ ^(A),

−v_(i) ^(k+1)

₁ ^(A),

−v_(i) ^(k+1)

₂ ^(A),

−q_(i, i+1) ^(k+1)v_(i) ^(k+1)

₁ ^(A), and

−q_(i, i+1) ^(k+1)v_(i) ^(k+1)

₂ ^(A), and the (i+1)^(th) participant provides

q_(i+1, i) ^(k+1)

₁ ^(A),

q_(i+1, i) ^(k+1)

₂ ^(A),

v_(i+1) ^(k+1)

₁ ^(A),

v_(i+1) ^(k+1)

₂ ^(A),

q_(i+1, i) ^(k+1)v_(i+1) ^(k+1)

₁ ^(A), and

q_(i+i, i) ^(k+1)v_(i) ^(k+1)

₂ ^(A). The i^(th) participant sends

q_(i, i+1) ^(k+1)

₂ ^(A),

−v_(i) ^(k+1)

₂ ^(A) and

−q_(i, i+1) ^(k+1)v_(i) ^(k+1)

₂ ^(A) to the (i+1)^(th) participant. The (i+1)^(th) participant sends

q_(i+1, i) ^(k+1)

₁ ^(A),

v_(i+1) ^(k+1)

₁ ^(A) and

q_(i+1, i) ^(k+1)v_(i+1) ^(k+1)

₁ ^(A) to the i^(th) participant. The i^(th) participant locallycomputes

−q_(i+1, i) ^(k+1)v_(i) ^(k+1)

₁ ^(A) and

q_(i, i+1) ^(k+1)v_(i+1) ^(k+1)

₁ ^(A) to finally determine the value of (q_(i, i+1) ^(k+1)+q_(i+1)^(k+1))(v_(i+1) ^(k+1)−v_(i) ^(k+1)) in the shared form as

(q_(i, i+1) ^(k+1)+q_(i+1, i) ^(k+1))(v_(i+1) ^(k+1))

₁ ^(A)=

q_(i, i+1) ^(k+1)v_(i+1) ^(k+1)

₁ ^(A)+

q_(i+1, i) ^(k+1)v_(i+1) ^(k+1)

₁ ^(A)+

−q_(i+1, i) ^(k+1)v_(i) ^(k+1)

₁ ^(A)+

−q_(i, i+1) ^(k+1)v_(i) ^(k+1)

₁ ^(A). Similarly, the (i+1)^(th) participant computes

(q_(i, i+1) ^(k+1)+q_(i+1, i) ^(k+1))(v_(i+1) ^(k+1)−v_(i) ^(k+1))

₂ ^(A)=

q_(i, i+1) ^(k+1)v_(i+1) ^(k+1)

₂ ^(A)+

q_(i+1, 1) ^(k+1)v_(i+1) ^(k+1)

₂ ^(A)+

−q_(i+1, i) ^(k+1),v_(i) ^(k+1)

₂ ^(A)+

−q_(i, i+1) ^(k+1)v_(i) ^(k+1)

₂ ^(A).

Step 6: the interacting participating parties reconstruct the secret(q_(i, i+1) ^(k)+q_(i+1, i) ^(k)) (v_(i+1) ^(k)−v_(i) ^(k)) as(q_(i, i+1) ^(k)+q_(i+1, i) ^(k)) (v_(i+1) ^(k)−v_(i) ^(k))=

(q_(i, i+1) ^(k)+q_(i+1, i) ^(k))(v_(i+1) ^(k)−v_(i) ^(k))

₁ ^(A)+

(q_(i, i+1) ^(k)+q_(i+1, i) ^(k))(v_(i+1) ^(k)−v_(i) ^(k))

₂ ^(A), and compute λ_(i, i+1) ^(k+1)=λ_(i, i+1) ^(k)+ρ_(i, i+1)^(k+1)(v_(i) ^(k+1)−v_(i+1) ^(k+1)) to update λ_(i, i+1) ^(k+1).

Step 7: according to a set threshold E, when a value of the objectivefunction at a current iteration minus a value of the objective functionat a previous iteration is less than the threshold, it is determinedthat the convergence is reached. Otherwise, returning to step 3 tocontinue the iteration.

Step 8: a training result is output.

The effectiveness of the present invention will be further describedbelow in conjunction with experiments.

1. Experimental Conditions

The experiment is simulated under Ubuntu-18.04.1, and the function ofsecure multi-party computation is implemented by using an ABY framework.The privacy-preserving scheme is implemented by c++.

2. Experimental Results and Analysis

In the present invention, Ubuntu is selected for simulation. The MNISTdata set and the breast cancer data set are selected for a test. 2, 3,4, 5, and 6 nodes are selected to perform a horizontal distributionexperiment and a vertical distribution experiment, respectively. In thesimulation experiment, the classification accuracy of the support vectormachine is 98%.

In the experiment, the established network communication model faces thethreat of data privacy leakage. Different users collaborate to train thesupport vector machine, and the intermediate state of the interactionduring the training process will leak privacy information such asgradient and objective function. The development of distributedscenarios brings an increasing amount of data. In order to break datasilos and carry out certain collaborative training scenarios, a feasibleprivacy-preserving method is indispensable. The prior distributedsupport vector machines have mutually reciprocal shortcomings betweencomputational overhead and security. That is, a high-security method hasthe problem of high computational overhead, whereas a high-efficiencymethod has security issues. In addition, such methods must giveconsideration to both the machine learning scenario and the accuracy ofthe training results. In the present invention, the support vectormachine for privacy-preserving is trained by combining the ADMMalgorithm and the secret sharing. During the training process among theentities, the entities exchange part of the information divided bythemselves for collaborative training. The present invention is based onan honest-but-curious model, in which all participating entities do nottrust each other, and complete the training under the premise thatindividual information will not be leaked.

FIG. 5 illustrates the collaborative training of two nodes of a breastcancer data set, in which the classification accuracy of the supportvector machine based on the privacy-preserving ADMM algorithm in thehorizontal data distribution scenario is 98.2%, while the classificationaccuracy in the case where only one node uses the gradient descentmethod to solve the optimization problem is also 98%. Therefore, it canbe known that the method of the present invention providescryptographically strong and secure privacy-preserving without affectingthe accuracy of the training results.

FIG. 6 illustrates the collaborative training of the two nodes of thebreast cancer data set. The classification accuracy of the supportvector machine based on the privacy-preserving ADMM algorithm in thevertical data distribution scenario is 97.7%. When the verticaldistribution is applied to an actual scenario, different entities canhave data sets with different features, and collaborate to train aglobal classification model.

It should be noted that the embodiments of the present invention can beimplemented by hardware, software, or a combination of software andhardware. The hardware part can be implemented by dedicated logic. Thesoftware part can be stored in a memory, and the system can be executedby appropriate instructions, for example, the system can be executed bya microprocessor or dedicated hardware. Those of ordinary skill in theart can understand that the above-mentioned devices and methods can beimplemented by using computer-executable instructions and/or controlcodes included in a processor. Such codes are provided, for example, ona carrier medium such as a magnetic disk, compact disc (CD) or digitalvideo disk read-only memory (DVD-ROM), a programmable memory such as aread-only memory (firmware), or a data carrier such as an optical orelectronic signal carrier. The device and its modules of the presentinvention can be implemented by very large-scale integrated circuits orgate arrays, semiconductors such as logic chips and transistors, orprogrammable hardware devices such as field programmable gate arrays andprogrammable logic devices, and other hardware circuits. Optionally, thedevice and its modules of the present invention can be implemented bysoftware executed by various types of processors, or can be implementedby a combination of the hardware circuit and the software as mentionedabove, such as firmware.

The above only describes the specific embodiments of the presentinvention, but the scope of protection of the present invention is notlimited thereto. Any modifications, equivalent replacements,improvements and others made by any person skilled in the art within thetechnical scope disclosed in the present invention and the spirit andprinciple of the present invention shall fall within the scope ofprotection of the present invention.

What is claimed is:
 1. A distributed support vector machineprivacy-preserving method, comprising: dividing a secret through asecret sharing among a plurality of participating entities, iterativelyexchanging a part of information divided by the plurality ofparticipating entities, and solving sub-problems locally; performing aniteration until a convergence is reached to find a global optimalsolution; and in consideration of generality of the distributed supportvector machine privacy-preserving method, adopting the distributedsupport vector machine privacy-preserving method based on a verticaldata distribution and the distributed support vector machineprivacy-preserving method based on a horizontal data distribution,respectively; wherein the plurality of participating entities do nottrust each other, and the plurality of participating entities interactthrough a multi-party computation for local training.
 2. The distributedsupport vector machine privacy-preserving method according to claim 1,further comprising: step 1: establishing a network communicationenvironment with a plurality of data sources; step 2: choosing a supportvector machine scenario with a vertical distribution or a horizontaldistribution according to a data distribution of the plurality of datasources; step 3: allowing the plurality of participating entities tosolve the sub-problems locally; step 4: allowing the plurality ofparticipating entities to use a Boolean sharing to split a penaltyparameter and exchange a part of the penalty parameter with aneighboring node to update the penalty parameter; step 5: allowing theplurality of participating entities to use an arithmetic sharing tosplit an updated iterative variable and exchange a part of the updatediterative variable with the neighboring node to compute a Lagrangeparameter in a shared form; step 6, allowing the plurality ofparticipating entities to reconstruct the secret; step 7, returning tostep 3 if the iteration does not reach the convergence; and step 8,outputting a training result.
 3. The distributed support vector machineprivacy-preserving method according to claim 2, wherein an objectivefunction of the horizontal distribution and an objective function of thevertical distribution in step 2 are respectively:$\min_{v_{i}}{,{{\xi_{i}\frac{1}{2}{\sum\limits_{i = 1}^{N}{{v_{i}^{T}( {I_{D + 1} - \Pi_{D + 1}} )}v_{i}}}} + {{NC}{\sum\limits_{i = 1}^{N}{1_{i}^{T}\xi_{i}}}}}}$$s.t.\mspace{14mu}\{ \begin{matrix}{{{Y_{i}B_{i}V_{i}} \succcurlyeq {1_{i} - \xi_{i}}},{{\xi_{i} \succcurlyeq {0_{i}( {{i = 1},\ldots\mspace{14mu},N} )}};}} \\{{AV} = 0}\end{matrix} $ wherein v_(i)=[ω_(i) ^(T), b_(i)]^(T), V=[v₁ ^(T),. . . v_(N) ^(T)]^(T), B_(i)=[X_(i),1_(i) ^(T)], 1_(i)∈R^(1*M), X_(i) isan i^(th) participant and a j^(th) participant, X_(i)=[[x_(i1) ^(T), . .. x_(iM) ^(T)]^(T)], Y_(i)=diag(y_(i1), . . . , y_(iM)), y_(ij) is aj^(th) data entry and a corresponding label, ξ_(i)=[ξ_(i1), . . . ,ξ_(iM)], N is a number of participants, and M is a number of trainingset samples for each participant of the participants; and${\min{\sum\limits_{i = 1}^{N}{\frac{1}{2}{v_{i}^{T}( {I_{D_{i} + 1} - \Pi_{D_{i} + 1}} )}V_{i}}}} + {1_{M}^{T}( {1_{M} - {Y{\sum\limits_{i = 1}^{N}{B_{i}v_{i}}}}} )_{+}}$s.t.  z_(i) = B_(i)v_(i), i = 1, …  , N; wherein v_(i)=[ω_(i)^(T),b_(i)]^(T), B_(i)=[X_(i), 1_(M)], 1_(M)∈R^(M*1), Y_(i)=diag(y₁, . .. , y_(M)), y_(j) is the j^(th) data entry and the corresponding label.4. The distributed support vector machine privacy-preserving methodaccording to claim 2, wherein iterative processes of solving thesub-problems locally in step 3 are respectively: for the horizontal datadistribution:${\{ {v_{i}^{k + 1},\xi_{i}^{k + 1}} \} = {{\arg\;{\min_{v_{i},\xi_{i}}{L( {v_{i}\xi_{i}\rho^{k}\lambda^{k}} )}}} + {\frac{r_{i}}{2}{{v_{i} - v_{i}^{k}}}^{2}}}}, \rho^{k}arrow\rho^{k + 1} ,{{\lambda_{i,{i + 1}}^{k + 1} = {\lambda_{i,{i + 1}}^{k} + {\rho_{i,{i + 1}}^{k + 1}( {v_{i}^{k + 1} - v_{i + 1}^{k + 1}} )}}};}$and for the vertical data distribution:${v_{i}^{k + 1} = {{\arg\;{\min_{v_{i}}{f_{i}( {vi} )}}} + {\frac{\rho}{2}{{{B_{i}v_{i}} - {B_{i}v_{i}^{k}} - {\overset{¯}{z}}^{k} + {\overset{\_}{Bv}}^{k} + u^{k}}}^{2}}}},{{\overset{¯}{z}}^{k + 1} = {{\arg\;{\min_{\overset{¯}{Z}}{g( {N\overset{¯}{z}} )}}} + {\frac{N\;\rho}{2}{{\overset{¯}{z} - {\overset{\_}{Bv}}^{k + 1} - u^{k}}}^{2}}}},{u^{k + 1} = {u^{k} + {\overset{\_}{Bv}}^{k + 1} - {{\overset{¯}{z}}^{k + 1}.}}}$5. The distributed support vector machine privacy-preserving methodaccording to claim 2, wherein a method of using the Boolean sharing tosplit the penalty parameter in step 4 specifically comprises:considering ρ^(k)→ρ^(k+1) is a progressive increase and an upper boundis r_(i), obtaining a value of the penalty parameter ρ through acomparison to update the penalty parameter ρ, and dividing ρ_(i, i+1)^(k) into ρ_(i, i+1) ^(k)=q_(i, i+1) ^(k)+q_(i+1, i) ^(k) to securelycompute ρ_(i, i+1) ^(k); wherein an i^(th) participant providesq_(i, i+1) ^(k) and q_(i, i+1) ^(k+1), and an (i+1)^(th) participantprovides q_(i+1, i) ^(k) and q_(i+1, i) ^(k+1); comparing q_(i, i+1)^(k)+q_(i+1, i) ^(k) with q_(i, i+1) ^(k)+q_(i+1, i) ^(k+1) withoutexposing q_(i, i+1) ^(k), q_(i+1, i) ^(k), q_(i, i+1) ^(k+1), andq_(i+1, i) ^(k+1), converting each term into a Boolean type, andperforming a secure addition and the comparison by using a Yao's garbledcircuit.
 6. The distributed support vector machine privacy-preservingmethod according to claim 2, wherein a method of using the arithmeticsharing to split the penalty parameter in step 5 specifically comprises:in a (k+1)^(th) iteration, securely computing ρ_(i, i+1) ^(k+1)(v_(i+1)^(k+1)−v_(i) ^(k+1)) as (q_(i, i+1) ^(k+1)+q_(i+1, i) ^(k+1))(v_(i+1)^(k+1)−v_(i) ^(k+1)) by using Shamir's secret sharing, andarithmetically dividing each term, wherein an i^(th) participantprovides

q_(i, i+1) ^(k+1)

₁ ^(A),

q_(i, i+1) ^(k+1)

₂ ^(A),

−v_(i) ^(k+1)

₁ ^(A),

−v_(i) ^(k+1)

₂ ^(A),

−q_(i, i+1) ^(k+1)v_(i) ^(k+1)

₁ ^(A), and

−q_(i, i+1) ^(k+1)v_(i) ^(k+1)

₂ ^(A), an (i+1)^(th) participant provides

q_(i+1, i) ^(k+1)

₁ ^(A),

q_(i+1, i) ^(k+1)

₂ ^(A),

v_(i+1) ^(k+1)

₁ ^(A),

v_(i+1) ^(k+1)

₂ ^(A),

q_(i+1, i) ^(k+1)v_(i+1) ^(k+1)

₁ ^(A), and

q_(i+1) ^(k+1)v_(i) ^(k+1)

₂ ^(A), the i^(th) participant sends

q_(i, i+1) ^(k+1)

₂ ^(A),

−v_(i) ^(k+1)

₂ ^(A) and

−q_(i, i+1) ^(k+1)v_(i) ^(k+1)

₂ ^(A) to the (i+1)^(th) participant, the (i+1)^(th) participant sends

q_(i+1, i) ^(k+1)

₁ ^(A),

v_(i+1) ^(k+1)

₁ ^(A) and

q_(i+1, i) ^(k+1)v_(i+1) ^(k+1)

₁ ^(A) to the i^(th) participant, and the i^(th) participant locallycomputes

−q_(i+1) ^(k+1)v_(i) ^(k+1)

₁ ^(A) and

q_(i, i+1) ^(k+1)v_(i+1) ^(k+1)

₁ ^(A) to finally determine a value of (q_(i, i+1) ^(k+1)+q_(i+1, i)^(k+1))(v_(i+1) ^(k+1)−v_(i) ^(k+1)) in the shared form as

(q_(i, i+1) ^(k+1)+q_(i+1, i) ^(k+1))(v_(i+1) ^(k+1)−v_(i) ^(k+1)−_(i)^(k+1))

₁ ^(A)=

q_(i, i+1) ^(k+1)v_(i+1) ^(k+1)+

q_(i+1) ^(k+1)v_(i+1) ^(k+1)

₁ ^(A)+

−q_(i+1) ^(k+1)v_(i) ^(k+1)

₁ ^(A)+

−q_(i, i+1) ^(k+1)v_(i) ^(k+1)

₁ ^(A); and the (i+1)^(th) participant computes

(q_(i, i+1) ^(k+1)+q_(i+1, i) ^(k+1))(v_(i+1) ^(k+1)−v_(i) ^(k+1))

₂ ^(A)=

q_(i, i+1) ^(k+1)v_(i+1) ^(k+1)

₂ ^(A)+

q_(i+1, i) ^(k+1)v_(i+1) ^(k+1)

₂ ^(A)+

−q_(i+1, i) ^(k+1)v_(i) ^(k+1)

₂ ^(A)+

−q_(i, i+1) ^(k+1)v_(i) ^(k+1)

₂ ^(A).
 7. The distributed support vector machine privacy-preservingmethod according to claim 2, wherein a method of reconstructing thesecret in step 6 specifically comprises: allowing the plurality ofparticipating entities to reconstruct the secret (q_(i, i+1)^(k)+q_(i+1, i) ^(k))(v_(i+1) ^(k)−v_(i) ^(k)) as (q_(i, i+1)^(k)+q_(i+1, i) ^(k))(v_(i+1) ^(k)−v_(i) ^(k))=

(q_(i, i+1) ^(k)+q_(i+1, i) ^(k))(v_(i+1) ^(k)−v_(i) ^(k))

₁ ^(A)+

(q_(i, i+1) ^(k)+q_(i+1, i) ^(k))(v_(i+1) ^(k)−v_(i) ^(k))

₂ ^(A), and compute λ_(i, i+1) ^(k+1)=λ_(i, i+1) ^(k)+ρ_(i, i+1)^(k+1)(v_(i) ^(k+1)−v_(i+1) ^(k+1)) to update the Lagrange parameter λ.8. A computer-readable storage medium, wherein a computer program isstored in the computer-readable storage medium, and when the computerprogram is executed by a processor, the processor executes the followingsteps: dividing a secret through a secret sharing among a plurality ofparticipating entities, iteratively exchanging a part of informationdivided by the plurality of participating entities, and solvingsub-problems locally; performing an iteration until a convergence isreached to find a global optimal solution; in consideration ofgenerality of a distributed support vector machine privacy-preservingmethod, adopting the distributed support vector machineprivacy-preserving method based on a vertical data distribution and thedistributed support vector machine privacy-preserving method based on ahorizontal data distribution, respectively; wherein the plurality ofparticipating entities do not trust each other, and the plurality ofparticipating entities interact through a multi-party computation forlocal training.
 9. A distributed support vector machineprivacy-preserving system for implementing the distributed supportvector machine privacy-preserving method according to claim 1,comprising: an information preprocessing module, configured for dividingthe secret through the secret sharing among the plurality ofparticipating entities, iteratively exchanging the part of theinformation divided by the plurality of participating entities, andsolving the sub-problems locally; an information iterative processingmodule, configured for performing the iteration until the convergence isreached to find the global optimal solution; and a privacy-preservingmodule, configured for adopting the distributed support vector machineprivacy-preserving method based on the vertical data distribution andthe distributed support vector machine privacy-preserving method basedon the horizontal data distribution, respectively; wherein the pluralityof participating entities do not trust each other, and the plurality ofparticipating entities interact through the multi-party computation forthe local training.
 10. A distributed support vector machine forimplementing the distributed support vector machine privacy-preservingmethod according to claim
 1. 11. The distributed support vector machineprivacy-preserving system according to claim 9, the distributed supportvector machine privacy-preserving method further comprises: step 1:establishing a network communication environment with a plurality ofdata sources; step 2: choosing a support vector machine scenario with avertical distribution or a horizontal distribution according to a datadistribution of the plurality of data sources; step 3: allowing theplurality of participating entities to solve the sub-problems locally;step 4: allowing the plurality of participating entities to use aBoolean sharing to split a penalty parameter and exchange a part of thepenalty parameter with a neighboring node to update the penaltyparameter; step 5: allowing the plurality of participating entities touse an arithmetic sharing to split an updated iterative variable andexchange a part of the updated iterative variable with the neighboringnode to compute a Lagrange parameter in a shared form; step 6, allowingthe plurality of participating entities to reconstruct the secret; step7, returning to step 3 if the iteration does not reach the convergence;and step 8, outputting a training result.
 12. The distributed supportvector machine privacy-preserving system according to claim 11, whereinan objective function of the horizontal distribution and an objectivefunction of the vertical distribution in step 2 are respectively:$\min_{v_{i}}{,{{\xi_{i}\frac{1}{2}{\sum\limits_{i = 1}^{N}{{v_{i}^{T}( {l_{D + 1} - \Pi_{D + 1}} )}v_{i}}}} + {NC{\sum\limits_{i = 1}^{N}{1_{i}^{T}\zeta_{i}}}}}}$$s.t.\mspace{14mu}\{ \begin{matrix}{{{Y_{i}B_{i}V_{i}} \succcurlyeq {1_{i} - \xi_{i}}},{{\xi_{i} \succcurlyeq {0_{i}( {{i = 1},\ldots\mspace{14mu},N} )}};}} \\{{AV} = 0}\end{matrix} $ wherein v_(i)=[(ω_(i) ^(T),b_(i)]^(T), V=[v₁ ^(T),. . . v_(N) ^(T)]^(T), B_(i)=[X_(i), 1_(i) ^(T)], 1_(i)∈R^(1*M), X_(i)is an i^(th) participant and a j^(th) participant, X_(i)=[[x_(i1) ^(T),. . . x_(iM) ^(T)]^(T)], Y_(i)=diag(y_(i1), . . . , y_(iM)), y_(ij) is aj^(th) data entry and a corresponding label, ξ_(i)=[ξ_(i1), . . . ,ξ_(iM)], N is a number of participants, and M is a number of trainingset samples for each participant of the participants; and${\min{\sum\limits_{i = 1}^{N}{\frac{1}{2}{v_{i}^{T}( {I_{D_{i} + 1} - \Pi_{D_{i} + 1}} )}V_{i}}}} + {1_{M}^{T}( {1_{M} - {Y{\sum\limits_{i = 1}^{N}{B_{i}v_{i}}}}} )_{+}}$s.t.  z_(i) = B_(i)v_(i), i = 1, …  , N; wherein v_(i)=[ω_(i)^(T),b_(i)]^(T), B_(i)=[X_(i),1_(M)], 1_(M)∈R^(M*1), Y_(i)=diag(y₁, . .. , y_(M)), y_(j) is the j^(th) data entry and the corresponding label.13. The distributed support vector machine privacy-preserving systemaccording to claim 11, wherein iterative processes of solving thesub-problems locally in step 3 are respectively: for the horizontal datadistribution:${\{ {v_{i}^{k + 1},\xi_{i}^{k + 1}} \} = {{\arg\;{\min_{v_{i},\xi_{i}}{L( {v_{i}\xi_{i}\rho^{k}\lambda^{k}} )}}} + {\frac{r_{i}}{2}{{v_{i} - v_{i}^{k}}}^{2}}}}, \rho^{k}arrow\rho^{k + 1} ,{{\lambda_{i,{i + 1}}^{k + 1} = {\lambda_{i,{i + 1}}^{k} + {\rho_{i,{i + 1}}^{k + 1}( {v_{i}^{k + 1} - v_{i + 1}^{k + 1}} )}}};}$and for the vertical data distribution:${v_{i}^{k + 1} = {{\arg\;{\min_{v_{i}}{f_{i}( v_{i} )}}} + {\frac{\rho}{2}{{{B_{i}v_{i}} - {B_{i}v_{i}^{k}} - {\overset{\_}{z}}^{k} + {\overset{\_}{Bv}}^{k} + u^{k}}}^{2}}}},{{\overset{\_}{z}}^{k + 1} = {{\arg\;{\min_{\overset{\_}{z}}{g( {N\;\overset{\_}{z}} )}}} + {\frac{N\;\rho}{2}{{\overset{\_}{z} - {\overset{\_}{Bv}}^{k + 1} - u^{k}}}^{2}}}},{u^{k + 1} = {u^{k} + {\overset{\_}{Bv}}^{k + 1} - {{\overset{\_}{z}}^{k + 1}.}}}$14. The distributed support vector machine privacy-preserving systemaccording to claim 11, wherein a method of using the Boolean sharing tosplit the penalty parameter in step 4 specifically comprises:considering ρ^(k)→ρ^(k+1) is a progressive increase and an upper boundis r_(i), obtaining a value of the penalty parameter ρ through acomparison to update the penalty parameter ρ, and dividing ρ_(i, i+1)^(k) into ρ_(i, i+1) ^(k)=q_(i, i+1) ^(k)+q_(i+1, i) ^(k) to securelycompute ρ_(i, i+1) ^(k); wherein an i^(th) participant providesq_(i, i+1) ^(k) and q_(i, i+1) ^(k+1), and an (i+1)^(th) participantprovides q_(i+1, i) ^(k) and q_(i+1, i) ^(k+1); comparing q_(i, i+1)^(k)+q_(i+1, i) ^(k) with q_(i, i+1) ⁺¹+q_(i+1, i) ^(k+1) withoutexposing q_(i, i+1) ^(k), q_(i+1, i) ^(k), q_(i, i+1) ^(k+1), andq_(i+1, i) ^(k+1), converting each term into a Boolean type, andperforming a secure addition and the comparison by using a Yao's garbledcircuit.
 15. The distributed support vector machine privacy-preservingsystem according to claim 11, wherein a method of using the arithmeticsharing to split the penalty parameter in step 5 specifically comprises:in a (k+1)^(th) iteration, securely computing ρ_(i, i+1) ^(k+1)(v_(i+1)^(k+1)−v_(i) ^(k+1)) as (q_(i, i+1) ^(k+1)+q_(i+1, i) ^(k+1))(v_(i+1)^(k+1)−v_(i) ^(k+1)) by using Shamir's secret sharing, andarithmetically dividing each term, wherein an i^(th) participantprovides

q_(i, i+1) ^(k+1)

₁ ^(A),

q_(i, i+1) ^(k+1)

₂ ^(A),

−v_(i) ^(k+1)

₁ ^(A),

−v_(i) ^(k+1)

₂ ^(A),

−q_(i, i+1) ^(k+1)v_(i) ^(k+1)

₁ ^(A), and

q_(i, i+1) ^(k+1)v_(i) ^(k+1)

₂ ^(A), an (i+1)^(th) participant provides

q_(i+1, i) ^(k+1)

₁ ^(A),

q_(i+1, i) ^(k+1)

₂ ^(A),

v_(i+1) ^(k+1)

₁ ^(A),

v_(i+1) ^(k+1)

₂ ^(A),

q_(i+1, i) ^(k+1)v_(i+1) ^(k+1)

₁ ^(A) and

q_(i+1, i) ^(k+1)v_(i) ^(k+1)

₂ ^(A), the i^(th) participant sends

q_(i, i+1) ^(k+1)

₂ ^(A),

−v_(i) ^(k+1)

₂ ^(A) and

−q_(i) ^(k+1)v_(i) ^(k+1)

₂ ^(A) to the (i+1)^(th) participant, the (i+1)^(th) participant sends

q_(i+1, i) ^(k+1)

₁ ^(A),

v_(i+1) ^(k+1)

₁ ^(A) and

q_(i+1) ^(k+1)v_(i+1) ^(k+1) to the i^(th) participant, and the i^(th)participant locally computes

−q_(i+1, i) ^(k+1)v_(i) ^(k+1)

₁ ^(A) and

q_(i, i+1) ^(k+1)v_(i+1) ^(k+1)

₁ ^(A) to finally determine a value of (q_(i, i+1) ^(k+1)+q_(i+1, i)^(k+1))(v_(i+1) ^(k+1)−v_(i) ^(k+1)) in the shared form as

(q_(i, i+1) ^(k+1)+q_(i+1, i) ^(k+1))(v_(i+1) ^(k+1)−v_(i) ^(k+1))

₁ ^(A)=

q_(i, i+1) ^(k+1)v_(i+1) ^(k+1)

₁ ^(A)+

q_(i+1, i) ^(k+1)v_(i+1) ^(k+1)

₁ ^(A)+

−q_(i+1, i) ^(k+1)v_(i) ^(k+1)

₁ ^(A)+

−q_(i, i+1) ^(k+1)v_(i) ^(k+1)

₁ ^(A); and the (i+1)^(th) participant computes

(q_(i, i+1) ^(k+1)+q_(i+1, i) ^(k+1))(v_(i+1) ^(k+1)−v_(i) ^(k+1))

₂ ^(A)=

q_(i, i+1) ^(k+1)v_(i+1) ^(k+1)

₂ ^(A)+

q_(i+1, i) ^(k+1)v_(i+1) ^(k+1)

₂ ^(A)+

−q_(i+1, i) ^(k+1)v_(i) ^(k+1)

₂ ^(A)+

−q_(i, i+1) ^(k+1)v_(i) ^(k+1)

₂ ^(A).
 16. The distributed support vector machine privacy-preservingsystem according to claim 11, wherein a method of reconstructing thesecret in step 6 specifically comprises: allowing the plurality ofparticipating entities to reconstruct the secret (q_(i, i+1)^(k)+q_(i+1, i) ^(k))(v_(i+1) ^(k)−v_(i) ^(k)) as (q_(i, i+1)^(k)+q_(i+1, i) ^(k))(v_(i+1) ^(k)−v_(i) ^(k))=

(q_(i, i+1) ^(k)+q_(i+1, i) ^(k))(v_(i+1) ^(k)−v_(i) ^(k))

₁ ^(A)+

(q_(i, i+1) ^(k)+q_(i+1, i) ^(k))(v_(i+1) ^(k)−v_(i) ^(k))

₂ ^(A), and compute λ_(i, i+1) ^(k+1)=λ_(i, i+1) ^(k)+ρ_(i, i+1)^(k+1)(v_(i) ^(k+1)−v_(i+1) ^(k+1)) to update the Lagrange parameter λ.17. The distributed support vector machine according to claim 10, thedistributed support vector machine privacy-preserving method furthercomprises: step 1: establishing a network communication environment witha plurality of data sources; step 2: choosing a support vector machinescenario with a vertical distribution or a horizontal distributionaccording to a data distribution of the plurality of data sources; step3: allowing the plurality of participating entities to solve thesub-problems locally; step 4: allowing the plurality of participatingentities to use a Boolean sharing to split a penalty parameter andexchange a part of the penalty parameter with a neighboring node toupdate the penalty parameter; step 5: allowing the plurality ofparticipating entities to use an arithmetic sharing to split an updatediterative variable and exchange a part of the updated iterative variablewith the neighboring node to compute a Lagrange parameter in a sharedform; step 6, allowing the plurality of participating entities toreconstruct the secret; step 7, returning to step 3 if the iterationdoes not reach the convergence; and step 8, outputting a trainingresult.
 18. The distributed support vector machine according to claim17, wherein an objective function of the horizontal distribution and anobjective function of the vertical distribution in step 2 arerespectively:$\min_{v_{i}}{,{{\xi_{i}\frac{1}{2}{\sum\limits_{i = 1}^{N}{{v_{i}^{T}( {I_{D + 1} - \Pi_{D + 1}} )}v_{i}}}} + {{NC}{\sum\limits_{i = 1}^{N}{1_{i}^{T}\xi_{i}}}}}}$$s.t.\mspace{14mu}\{ \begin{matrix}{{{Y_{i}B_{i}V_{i}} \succcurlyeq {1_{i} - \xi_{i}}},{{\xi_{i} \succcurlyeq {0_{i}( {{i = 1},\ldots\mspace{14mu},N} )}};}} \\{{AV} = 0}\end{matrix} $ wherein v_(i)=[ω_(i) ^(T),b_(i)]^(T), V=[v₁ ^(T),. . . v_(N) ^(T)]^(T), B_(i)=[X_(i),1_(i) ^(T)], 1_(i)∈R^(1*M), X_(i) isan i^(th) participant and a j^(th) participant, X_(i)=[[x_(i1) ^(T), . .. x_(iM) ^(T)]^(T)], Y_(i)=diag(y_(i1), . . . , y_(iM)), y_(ij) is aj^(th) data entry and a corresponding label, ξ_(i)=[ξ_(i1), . . . ,ξ_(iM)], N is a number of participants, and M is a number of trainingset samples for each participant of the participants; and${\min{\sum\limits_{i = 1}^{N}{\frac{1}{2}{v_{i}^{T}( {I_{D_{i} + 1} - \Pi_{D_{i} + 1}} )}V_{i}}}} + {1_{M}^{T}( {1_{M} - {Y{\sum\limits_{i = 1}^{N}{B_{i}v_{i}}}}} )_{+}}$s.t.  z_(i) = B_(i)v_(i), i = 1, …  , N; wherein v_(i)=[ω_(i)^(T),b_(i)]^(T), B_(i)=[X_(i),1_(M)], 1_(M)∈R^(M*1), Y_(i)=diag(y₁, . .. , y_(M)), y_(j) is the j^(th) data entry and the corresponding label.19. The distributed support vector machine according to claim 17,wherein iterative processes of solving the sub-problems locally in step3 are respectively: for the horizontal data distribution:${\{ {v_{i}^{k + 1},\xi_{i}^{k + 1}} \} = {{\arg\;{\min_{v_{i},\xi_{i}}{L( {v_{i}\xi_{i}\rho^{k}\lambda^{k}} )}}} + {\frac{r_{i}}{2}{{v_{i} - v_{i}^{k}}}^{2}}}}, \rho^{k}arrow\rho^{k + 1} ,{{\lambda_{i,{i + 1}}^{k + 1} = {\lambda_{i,{i + 1}}^{k} + {\rho_{i,{i + 1}}^{k + 1}( {v_{i}^{k + 1} - v_{i + 1}^{k + 1}} )}}};}$and for the vertical data distribution:${v_{i}^{k + 1} = {{\arg\;{\min_{v_{i}}{f_{i}( v_{i} )}}} + {\frac{\rho}{2}{{{B_{i}v_{i}} - {B_{i}v_{i}^{k}} - {\overset{¯}{z}}^{k} + {\overset{\_}{Bv}}^{k} + u^{k}}}^{2}}}},{{\overset{¯}{z}}^{k + 1} = {{\arg\;{\min_{\overset{¯}{Z}}{g( {N\overset{¯}{z}} )}}} + {\frac{N\;\rho}{2}{{\overset{¯}{z} - {\overset{\_}{Bv}}^{k + 1} - u^{k}}}^{2}}}},{u^{k + 1} = {u^{k} + {\overset{\_}{Bv}}^{k + 1} - {{\overset{¯}{z}}^{k + 1}.}}}$20. The distributed support vector machine according to claim 17,wherein a method of using the Boolean sharing to split the penaltyparameter in step 4 specifically comprises: considering ρ^(k)→ρ^(k+1) isa progressive increase and an upper bound is r_(i), obtaining a value ofthe penalty parameter ρ through a comparison to update the penaltyparameter ρ, and dividing ρ_(i, i+1) ^(k) into ρ_(i, i+1)^(k)=q_(i, i+1) ^(k)+q_(i+1, i) ^(k) to securely compute ρ_(i, i+1)^(k); wherein an i^(th) participant provides q_(i, i+1) ^(k) andq_(i, i+1) ^(k+1), and an (i+1)^(th) participant provides q_(i+1, i)^(k) and q_(i+1, i) ^(k+1); comparing q_(i, i+1) ^(k)+q_(i+1, i) ^(k)with q_(i, i+1) ^(k+1)+q_(i+1, i) ^(k+1) without exposing q_(i, i+1)^(k), q_(i+1, i) ^(k), q_(i, i+1) ^(k+1), and q_(i+1, i) ^(k+1),converting each term into a Boolean type, and performing a secureaddition and the comparison by using a Yao's garbled circuit.